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FOREWORD 

The committee appointed to advise on this project came to 
the unanimous conclusion that time is more than ripe to improve pro- 
duct selection procedures in American education. A massive expansion 
in the use of the technology which society puts at the school's dis- 
posal is inevitable if we are to keep our schools efficient. It is 
urgent that the large sums of money required to implement technologi- 
cal developments be carefully and wisely administered , and that 
insofar as possible teachers and students find themselves equipped for 
the experience of learning. The committee members were able to add to 
one another's cautionary tales concerning instructional equipment for 
which no materials are available , or which have been offered to edu- 
cators before being sufficiently tried out by their developers. The 
committee members also exchanged Caveats concerning materials which 
require highly trained operating personnel , or which add significant 
amounts to schools' operating and maintenance budgets. They were also 
aware of books and learning systems and kits which operate effectively 
only when the teacher can deal with one or at most three students at a 
time. In addition they were concerned about those materials which de- 
pend for full usefulness upon access to sources not readily available, 
or materials which are attractively packaged or illustrated but which 
turn out to have little substance. 

The selection practices which might lead to serious errors of 
purchase from the expanded materials market are unconsciously revealed 
in reported comments from persons familiar with how materials are 

* i i 

in 



o 

uc 



mm 



! ! • - v • 



BPS 






IV 



chosen. It seems, for instance, that selectors sometimes fail to 
investigate claims made for products. Sometimes they select from 
among the first few items offered, either because they are unaware of 
more appropriate options or because there is no time to look further. 

Or purchasers may select in too small a context, without sufficient 
information about the other materials students will be using and with 
no idea of the possibilities for interaction among various materials. 
They may buy in blind faith that a device, because it is offered to 
schools, can be operated safely by teachers and students. Sometimes 
the purchaser is seduced by the ingenuity or popularity of a new de- 
vice without proper regard to the needs for special training for 
teachers using it or for additional personnel to operate and maintain it 
The reports received in the survey were not reassuring that 
such selection "accidents" cannot continue to happen. Particularly it 
was clear that even if local selectors were always aware of all the 
types of information they need before making a choice, it would be be- 
yond the resources of most local districts to develop such information. 

A need for central sources of information made itself felt. Accordingly 
the committee, at its last meeting, agreed to the following propositions 
In the matter of the selection of educational materials, 
every state education authority has a responsibility it 
cannot abdicate to see that essential information reaches 



local schools. 



The information should be gathered from, and shared among, 
various kinds of agencies such as local school systems, 
regional or state education authorities, and independent 
agencies, to avoid duplication of effort. 



The committee recognizes the necessity for documenting 
these findings in a report and for setting a framework 
for further action, including devising a means or speci- 
fying an agency to coordinate such action. 



This report is the first step towards the goal of stimulating 
improvement in selection practices. Other incentives and assistances 
will be forthcoming. Suggestions and offers of cooperation will be 
appreciated. 



The Advisory Committee : 

Edward G. Bernard (New York City) 
James A. Boula (Illinois) 
Lee E. Campion (New York State) 
Nile D. Coon (Pennsylvania) 
Robert Gerletti (Los Angeles County) 

Claude Hass (California) 
Guy Helmke (California) 
William C. Lewis (Delaware) 
George Lipscomb (Texas) 
Francis Pinkowski (New Jersey) 
David J. Rees (New York State) 
J. M. Shaver (North Carolina) 



PREFACE 



This is a report of existing evaluation practices employed in 
the selection of educational materials and equipment for use in elemen- 
tary and secondary schools. The U. S. Office of Education funded the 
major portion of the study as provided in Title V, Section* 505, of the 
Elementary and Secondary Education Act, under a contract with the New 
York State Education Department, which also supplied matching funds. 
Representatives of state education authorities in California, Delaware, 
Illinois, New Jersey, North Carolina, Pennsylvania, and Texas served as 
advisors, and the Educational Products Information Exchange (EPIE) 

Institute carried out the work of the survey under a subcontract. The 
ultimate aim of the study, in accordance with the provisions of Title V 
ESEA, is to determine how the state department of education may increase 
its usefulness to local agencies in the process of evaluating and select- 
ing educational products. 

Although the sponsors of this study were primarily interested in 
learning about the varying roles played by state departments in materials 
and equipment selection, the Advisory Committee (representatives of the 
eight cooperating state education authorities and of two large school 
systems) directed the EPIE Institute to report on any identifiable evalua- 
tion practices for the selection of instructional material and equipment, 
whether or not related to the activities of state departments of education. 
Given this charge and the necessarily short duration (six months) of the 
study, the following information-gathering procedures were adopted; 
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1. The Director of the Division of Educational Com- 
munications, New York State Education Department, wrote to 
the representative in each state of the Association of Chief 
State School Audio-Visual Officers, and to the coordinator 
for each state of activities under Title III of the National 
Defense Education Act, explaining the project and requesting 
a "telephone appointment." 

2. On the basis of replies, persons in each state 
were telephoned by members of the EPIE staff or by an 
Associate in Educational Communications, New York State 
Education Department. Copies of relevant reports, forms, 
and procedures were requested, and assistance was requested 
in finding others who might be able to help in the project. 

3. On the basis of replies, the Director of Educa- 
tional Communications, New York State Education Department, 
wrote to approximately seventy-five more persons, and the 
EPIE staff made some fifty additional telephone calls. 

4. On the basis of information received during con- 
sultations with university curriculum researchers, with 
school officials, and with organizations concerned with 
educational materials purchasers, the EPIE Institute staff 
wrote to about twenty additional persons. 

5. More than one hundred fifty reports or statements 
were received. Some twenty-five were not germane to this 
particular project. The others were read by the EPIE staff 



and coded as to nature of materials being evaluated, of 
persons making judgments, or procedures used, and other 
pertinent facts. In addition, approximately fifteen re- 
ports already in the possession of the EPIE institute were 
read and coded. 

6. The coded materials were used to prepare ex- 
amples of differing selection practices encountered during 
the survey. 

These information-gathering and summarizing activities resulted 
in a full (though not exhaustive) report. The scope of the report is the 
description of present practices in evaluation for selection and use — how 
school decision-makers decide which materials to use for instruction. 
Except that satisfaction (or lack of it) with previously used materials 
form part of the selection process, the results of other types of evalua- 
tion (programmatic, training, developmental) are not considered in this 
report. 

The information received in the course of the survey divides 
itself into three major categories: selection of textbooks, including 

any supplementary materials offered as part of the textbook program; 
selection of other supplementary materials — additional books, films, kits, 
exhibits and such items; and selection of equipment, which can range from 
buying an overhead projector to introducing instructional television into 
the classroom. The evidence is that the increased use of nonprint mate- 
rials has led to the development of processes and techniques for evalua- 
ting them — and their mediating devices — but they are still, for the most 



part, separated from and less elaborate and formal than textbook selec- 
tion processes. 

The survey results are presented as an array of concrete ex- 
amples of actual practices now being used across the country by state 
departments, local shcool systems, ESEA Title III and Special Education 
Materials Centers, regional media projects, the military, and religious 
educational systems. The information is presented in this concrete 
fashion with the hope that such a description of the present "state of 
the art" of material and equipment selection will cause all concerned 
with this important educational activity to reflect upon and act to im- 
prove upon both the efficiency and the effectiveness of existing evalua- 
tion and selection practices. State education departments in particular 
should find much in this document to cause them to rededicate themselves 
to fostering improvements in these practices at both the state and the 
local level. The persons responsible, for the programs described in many 
cases welcome further questions, which the EPIE Institute (386 Park 
Avenue South, New York, New York 10016) will be glad to transmit. 

The sponsoring agencies wish to acknowledge the excellent co- 
operation of those persons in state education departments throughout the 
country who cooperated with the project staff in supplying information 
either through correspondence or via telephone interviews. They are to 
be thanked particularly for their help in identifying nonstate-agency 
evaluation activities in their own and other states. The sponsors also 
wish to recognize the work of David Rees of the Division of Educational 
Communications, New York State Education Department, who served as 















xi 

liaison between his department and the EPIE Institute and as coordinator 
for Advisory Committee meetings. In particular they wish to acknowledge 
the contribution made at every phase of the study by Betty Braxton Preston, 
Director of Special Projects at the EPIE Institute, who took the major 
responsibility not only for collecting the survey information but also 
for producing from it a cohesive and serviceable report. 



Lee E. Campion 
New York State Education Department 

P. Kenneth Komoski 
The EPIE Institute 
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THE STATE OF THE ART 



It is not easy for the educator to grasp even in his imagina- 



tion the vast variety of instructional materials available today. To 



take textbooks alone, a state department of public instruction noted in 



1967 that there were forty-four hundred different textbook titles of- 



fered for sale in the state, six hundred and sixty of them in the area 



of social studies. When one adds the materials, devices, systems, and 



equipment available to supplement or supplant the textbook, it is a mat- 



ter of wonder and, all too often, bewilderment to the educational deci- 



sion maker. Materials are available to engage the student's eye or ear, 



or both, for manipulation, for experimentation, for interaction and re- 



inforcement — not only because electronic and other such advances have 



made these developments possible, but also primarily because the stu- 



dent's newly complex world makes it imperative to offer him more than 



text and teacher. 



In the face of this flood of materials, and of newly available 



funds to purchase them, educational decision makers have begun to devel- 



op techniques which will permit competent and satisfactory selections 



while avoiding waste and duplication of effort in the selection process, 



with what would appear to be, so far, limited success. There are egre- 



gious examples of careful and thoughtful approaches to the question of 



^Michigan Department of Education. A Report on the Treatment of 
Minorities in American History Textbooks , Lansing: Michigan Department 

of Education, 1968, Appendix A, no pagination. 
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selecting materials, but they are, unfortunately, far from widespread. 
Certainly they were late in developing. 

In all too few cases, school districts have banded together 
to share the costs and tasks of investigating the huge collection of 
educational products before deciding which to buy. Results are, for the 
most part, disseminated only within the area which undertook the coop- 
erative effort, though frequently their usefulness could be much more 
extensive than that. 

There is some recourse, in the new attempt to systematize the 
selection of materials, to check lists and guidelines — an appendix to 
this report is devoted to a review of such instruments and their place 
in the selection process. As will be seen, they may facilitate that 
process but they assuredly are not the cure for all selection ills, and 
when they lend an illusion of scientific precision they are the very 
opposite of useful. 

Also added to the usual examination or testing of materials or 
equipment are such practices as trial use of products in a classroom 
setting, formal meetings with producers of materials, and consultation 
with curriculum or subject-matter specialists. Particularly in deci- 
sions as to whether to add a book or a film to a library, educators make 
use of published reviews and annotated catalogs, not only for prelimi- 
nary screening but in the final decision. And published analyses by 
subject-matter experts of curriculum materials are just beginning to 



play a part in selection. 



An increasingly common selection practice, particularly for 
supplementary materials, is use of central collections of materials--an 
instructional— mater ials center, an educational— resources center, an edu- 
cational-media selection center, or similarly named facility — where one 
may examine materials and discuss them with a curriculum specialist. 
Educators fortunate enough to have access to such a center are spared 
considerable paperwork, save a great deal of time, and more than likely 
solve a space and storage problem. 

It is proper to mention here the basic work of the EPIE 
Institute, the nationwide educational information exchange which pre- 
pared this report. Comprehensive descriptions and evaluative reports on 
materials available for a given purpose, with information presented in 
comparable terms and, where possible, in tabular form to permit across- 
the-board comparisons and evaluations, are available from the Institute. 
Information reported includes critical analysis of content of material 
and evaluations based on user experience. Expansion is continuing of 
areas and materials covered, varieties of professional analysis provided, 
and base for collecting, classifying, and reporting user experiences. 
Consultative and training services are available as well as reports. 

To reiterate by way of summary — the users of these various 
plans, procedures, and aids are still far too few? many more need to be 
brought to the realization that selection plays a vital part in using 
educational materials effectively. And the few who are attempting sys- 
tematization are far from smug about the completeness and efficiency of 



their procedures as so far developed. They report numerous lacks and 
dissatisfactions. 

The lack of consistency in educators' judgments is a diffi- 
culty; requesting responses to specific questions helps in this connec- 
tion, and there is evidence that training in selection practices would 
help even more. Perhaps training, too, could assist in the problem of 
special point of view — educators ignore technical aspects sometimes; 
engineers forget educational considerations sometimes. The fact that 
persons selected to evaluate materials are not, in some cases, repre- 
sentative of the entire community for which they are selecting is 
another problem. One large city has established a procedure for ex- 
amination — after preliminary selection but before final selection — of 
all textbooks in light of criteria concerning the presentation of infor- 
mation about minority groups. 

Still another problem is that of significant differences among 
teachers in their approach to and use of materials, facts not easily 
known to or usable by those who must recommend materials. This problem 
is compounded by the lack so far of systematic information about users' 
experiences with materials. "Popularity" ratings are sometimes avail- 
able, but they are related only to frequency of use, not to conditions 
of use or effectiveness of results. 

Since teachers very often write or edit textbooks, selection 
of persons to review them sometimes presents problems. State statutes 
often make it explicit that no one shall evaluate textbooks during a 
period when he has any sort of business relationships with a publisher 
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who is submitting textbooks for evaluation. Similar prohibitions occur 
in some city regulations as well. 

A special aspect of the always troublesome time factor de- 
serves some attention. Extensive evaluation projects, particularly 
those to select textbooks, often consider materials on a rotating sched- 
ule — social studies one year, mathematics the next, for instance — since 
manpower and purchase money are far from unlimited. This can mean as 
much as a six-year gap in the consideration of a given type of mate- 
rials unless special arrangements are made. Similar problems in proj- 
ects to add supplementary education materials to collections are the 
administrative ones of scheduling evaluators and collating their 
evaluations. 

In the matter of materials, there can be other problems than 
the one of oversupply. A contradictory one is the sometimes -no ted 
absence of any material to fill a stated need — an indication perhaps of 
the desirability of closer cooperation between users and developers of 
materials. Inadequate or inaccurate descriptions of materials submit- 
ted for review can lead to waste of time; some programs include detailed 
specifications to producers, in part to combat just that sort of waste, 
at the risk, perhaps, of missing some new or innovative approaches to 
educational problems. 

Certain aspects of the problems of selection which came clear 
in the course of the survey, but which were not specifically mentioned 
by respondents, include the effect of legalities and red tape on the 
process. Not only are state laws sometimes specific about material 
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which must be covered in — or omitted from — textbooks in use in the state, 
but, as implied earlier, they may make it impossible, or at least very 
difficult, to change to a new, superior textbook until a stated period 
of adoption is over. Similarly, purchase regulations may necessitate 
purchase at less than the optimum time, with insufficient information on 
hand for wise selection. In cases, for instance, where budgetary re- 
quirements make it necessary to use up purchase funds before a deadline, 
money is sometimes spent, without much investigation, for materials 
which turn out to be useless. 

Perhaps the most distressing reports of all are those of no 
systematic evaluation programs at all because of lack of funds to sup- 
port them. The suspicion approaches a conviction that such saving is 
false economy. 

Each of the major types of educational product — textbooks, 
supplementary materials, and equipment— —presents a different selection 
pattern. The rest of this report is devoted to detailed descriptions, 
without comment by reporter or compiler, of programs which illustrate 
those patterns. It is hoped that they will give the reader a compre- 
hensive picture of the state of the art (it is hardly yet a science) of 



materials selection. 



TEXTBOOK SELECTION 



Typical procedures for textbook selection indicate recognition 
of the continued centrality and therefore importance of textbooks in 
the instructional picture — considerable time, effort, and money go into 
their selection. Much emphasis is put on the opinion of the experienced 
teacher; even in the very few cases where a state authority specifies 
one particular book for use throughout the state in a given class and 
subject, at least a few classroom teachers have had some say in selec- 
tion. 

In twenty-four states, textbooks are evaluated on a state level 
and a selection of books is adopted for use throughout the state; in the 
other states, a local authority is responsible for evaluation and selec- 
tion of textbooks. (In a few cases, the state adopts books for elemen- 
tary use and local units select books for secondary use.) That distinc- 
tion is an important one, but the variations within each of the two 
styles of selection are so great that practices in differently labeled 
states may not vary as much as the distinction implies. At least one 
adoption state lists almost every textbook available for any given sub- 
ject, another allows local school units to choose whether they will par- 
ticipate in the state textbook program, and several others permit choice 
of books outside the adopted list, with more-or-less easily obtained 
state approval. One state permits local districts to withdraw from the 
state-supported school system, and another permits schools to add to the 



state textbook list. 
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On the other hand, several nonadoption states have detailed 
statutory regulations governing local selection procedures, as well as 
prescribed courses of study. Others require that local systems list 
their selections with a state authority for his approval (and at least 
one of those states specifies the criteria against which local authori- 
ties must compare books) . Some states, too, specify how frequently and 
in what sequence local school authorities must select texts. 

In reviewing the procedures in use for selecting textbooks, it 

( 

is possible to isolate steps taken, in various combinations and sequence 
by selection committees. It may be useful to list those steps here, 
with the comment that no one procedure encountered encompasses all the 
steps, but that no example came forth which did not include examination 
and review of textbooks under consideration. 

1. Curriculum planning 

2. Development of criteria 

3. Examination and review of textbooks 

4. Meetings with textbook publishers' representatives 

5. Trial use of textbooks in classroom situations 

6. Consultation with curriculum and subject-matter 
specialists 

7. Submission of textbook choices to the public 

The examples of textbook-selection procedures described in the 
following pages show how several state and local authorities, working in 



hearsay report, received just before this report went to press, 
indicates that two school systems in the same nonadoption state add to 
this list of steps the solicitation of opinion of texts from students. 






9 




the framework of legal requirements and restraints, curriculum considera- 
tions, and budget and time limitations, combine and arrange these pro- 
cesses in order to evaluate textbooks. It will be noted that in several 
instances a description of local practices follows immediately, and 
refers specifically to, the description of procedures in the home state. 
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index No. 018: An Adoption State Enlists the Aid of the State 

Education Authority in Final Review 

After the special textbook committee makes its 
selections, the staff of the state education 
authority reviews the selections from a special 
point of view. 

I. The State Board of Education decides in which areas text- 

books will be adopted in a given year. Most books are 
adopted for a six -year period; they may be readopted. 

Also, textbooks may be adopted for new courses in which 
at least ten thousand students are enrolled throughout 
the state. Publishers are requested to notify the Board 
of books which they offer for adoption. 

II. The State Board of Education appoints a Textbook Com- 

mittee composed of fifteen teachers and administrators. 
Committee members during their tenure are state officials 
and take the oaths required of all state officials. 

III. The list of textbooks to be considered is published in 

newspapers throughout the state, and members of the Text- 
book Committee hear citizens regarding any objections to 
books listed. 

IV. Each Textbook Committee member meets with a representative 

of each publisher, with his chosen advisers (lay or pro- 
fessional specialists in a subject-matter field) present 
if he so desires. 

V. The Textbook Committee reviews books, with the help of 

its appointed advisers, and recommends to the State 

o 
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Commissioner of Education five books for each subject, 
or fewer if five suitable books are not available, but 
never fewer than two. 



all recommended books for the Commissioner of Education, 
to see if they are suitable, present factual information, 
and adhere to American ideals. 

The Commissioner may not add titles, but may remove them, 
so long as two per subject remain. Publishers or other 
citizens may request hearings if dissatisfied with the 
Commissioner's decisions. 



Board of Education, which holds a meeting to hear any 
appeals from the recommendations of the Commissioner, to 
adopt textbooks, and to approve contracts for their pur- 



VI. 



Members of the state education authority staff review 



VIII. 



The list of recommended books is presented to the State 
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Index No. 101: A City in an Adoption State (018) Selects 

Books from the State List 



I. 



Every teacher's opinion is sought as the city 
decides among the offerings on the state list. 



A. 



B. 



C. 



D. 



A Textbook Adoption Committee is appointed, 



consisting, for secondary book selection, of 



one teacher from each school for each subject 



being considered, and for elementary schools, 



one teacher and the principal from twenty rep- 



resentative elementary schools. The Adoption 



Committee operates in subcommittees, according 



to subject and grade. 



Each subcommittee develops criteria for evalu- 



ating books and prepares evaluation check lists 



The subcommittees meet with representatives of 



textbook publishers to hear presentations con- 



cerning their books. 



Each school representative has every teacher in 



his school who teaches a subject or grade under 



consideration evaluate and complete check lists 



for the books on the state list. The school 



representative summarizes the evaluations and 



indicates first, second, and third choices of 



the teachers in his school. 
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E. Each subcommittee then meets to make a final 

decision and to recommend a book for adoption 
to the Board of School Trustees. 

The Board of School Trustees, on the basis of the recom- 
mendations, adopts textbooks for use in schools through- 
out the city for the period specified by the state educa- 
tion authority. 
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Index No. 097: A Nonadoption State Surveys the Treatment 

of Minorities in State-Used Textbooks 

A state department of education conducts a study which 
produces guidelines to assist in the selecting of text- 
books which, as required by state law, fairly recognize 
the achievements and contributions of all racial and 
ethnic groups. 

I. In response to a legislative act requiring the State 

Superintendent of Schools to survey the social -studies 
texts in use in the state to determine whether they in- 
clude recognition of the achievements of racial and 
ethnic groups, a small professional committee was formed 
in 1967 to report to the State Superintendent. The com- 
mittee narrowed its task to consider only American his- 
tory texts, as being used by large numbers of students 
and as closely related to the intent of the legislation 
under which they had been appointed. 

II. A survey of a twenty-five percent representative sample 

of school districts in the state, supplemented by informa 
tion from publishers about textbook adoptions, resulted 
in a list of nine secondary and three elementary American 
history texts in common use throughout the state. 

III. A committee of professional historians was asked to nomi- 

nate six other professional historians to serve on an 
American History Review Panel. Each member of the panel 
was asked to review in detail two of the twelve commonly 
used books? reviewers and books were paired at random. 
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The reviewers' criticisms confirmed apparent doubts 
about the adequacy of the textbooks from the standpoint 
of their treatment of minorities, especially Negroes. 
Indeed, their overall judgment of the books in use 
amounted to condemnation, though there were favorable 
comments about each book and they were not all judged to 
be equally inadequate. The reviews supplied some of the 
basis for the development of detailed guidelines for use 
in evaluating textbooks on the dimension of treatment of 
minorities. The guidelines cover four major considera- 
tions: historical accuracy, realistic treatment of the 

history of minorities, the concept of race, and the total 
effect of the textbook's treatment of minorities. 

IV. The committee recommends to local school authorities 

selection of textbooks in terms of the criteria, supple- 
menting presently used inadequate books with instructional 
materials that more nearly meet the criteria, and pre- 
paring staffs for use of the criteria in judging all 



instructional materials. 



Index No. 013; A City in a Nonadoption State Considers Treatment 

of Minorities in Selecting Textbooks 

Two years before the passage in 1966 of a state law 
(see Index No. 097) requiring that textbooks deal 
fairly with the treatment of minorities , this city 
instituted mechanisms for dealing with such treatment 
in the textbooks it adopts. 

I. The textbook adoption procedure for the city has normally 

consisted of these five steps: 

A. Decision to replace currently adopted books. 
(Books are usually adopted for a five-year 

period. ) 

i 

B. ( Request for examples of appropriate books from 

publishers. 

C. Appointment of selection committees composed 
of teachers, supervisors, and administrators 
for each subject and grade. 

D. Action by selection committees to establish 
criteria, meet with publishers' representatives, 
examine and judge books, and make recommenda- 
tions, with reasons therefor reported, to the 
Superintendent of Schools. 

E. Recommendation by the Superintendent to the 
Board of Education for adoption of textbooks. 

II. In 1964, by action of the Board of Education, a sixth 

step was instituted. After selection committees have 
made their recommendations, the Department of Intergroup 
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Relations of the Division of School-Community Relations 
makes careful analyses of the listed books in terms of 
twenty criteria developed by the Department. The cri- 
teria require positive (rather than mere absence of 
negative) treatment of minority groups. The Intergroup 
Relations Department then makes recommendations to the 
Superintendent, who makes use of the recommendations in 

his determination of what textbooks should be presented 

* 

to the Board of Education for adoption. 
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No. 084: The State Board of Education Directs Adoption 

of Elementary-Level Textbooks 

A very large committee makes recommendations of text- 
books to be adopted on a five-year rotating schedule. 

Each of the nine State Board of Education members ap- 
points a State Evaluation chairman and with his help 
selects from four to seven additional members to serve 
with the chairman on the committee. The members and 
chairmen must be experienced and active educators in 
the state, not directly or indirectly connected with 
publishing houses which will submit programs to the com- 
mittees. At least half the members of each committee 
must be teachers, and wide cross-sectional representation 
is recommended. 

At an organizational meeting, the nine State Evaluation 
Chairmens 

A. Determine cooperatively the objectives of 
adoption 

B. Establish time schedules for evaluation 

C. Develop criteria to be used for the evaluation 

D. Recommend categories of subject area 

E. Define methods of balloting and procedures to 
be followed 

The Director of Elementary Education of the State Depart- 



ment of Public Instruction makes arrangements for formal 



presentations to the State Committee by publishers' 
representatives. Time allotments are equitable and can- 
not be exceeded. 

Committees evaluate all materials submitted by publishers, 
applying the criteria developed by the State Evaluation 
Chairmen. Each member ranks each item considered in 
order of preference and a committee consensus of prefer- 
ence is developed. 

Each committee notifies the State Superintendent of 
Public Instruction which items it considers should be 
eliminated from further consideration. Vote of five of 
the nine committees effects such elimination. 

The State Evaluation Chairmen consolidate the recommenda- 
tions of the nine committees and submit the resulting 
report to the State Board of Education. 

The State Board votes on adoption at an open meeting; 
tallies of votes must be visible to spectators. From 
three to five textbooks per grade and subject are adopted 
for a five-year period. 
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009: A City Located in a State which Adopts Elementary 

Textbooks (084) Adopts Secondary Texts 

Committees, sometimes including townspeople, recom- 
mend books to be put on a multiple list, on a four- 
year rotating schedule. 

Each year a Selection Committee is appointed for each sub- 
ject area in which adoptions will be made, with the school 
district's subject-area consultants as chairmen, and de- 
partment heads and other teachers from all ten schools as 
members, it is possible also to include interested com- 
munity representatives. 

During the summer textbook publishers are informed of the 
areas in which adoptions will be made, procedures for 
making presentations about books to committee members, and 
procedures for submitting examples of textbooks for evalu- 
ation. 
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IV. 



From October to February, Selection Committees make com- 
plete and thorough examinations of each book submitted, 
according to general criteria supplied by the city and 
more specific ones developed by the committee. Procedures 
for evaluation may include: 

A. Consultation with specialists 

B. Trial use in classrooms 

C. Meetings with publisher's representatives 

In April the committees make final recommendation to the 
Assistant Superintendent for Instruction of up to five 
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books per course of study for adoption for the next four 

// 

years. The Assistant Superintendent for Instruction ad- 
vises the Board of Education, which makes final adoptions. 
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Index No. 005: A Nonadoption State Develops a 

Comprehensive Curriculum Aid 

Detailed analyses of twenty elementary school mathe- 
matics texts are available to any school in the state 
as tools in selection, in a project sponsored jointly 
by the U.S. Office of Education and a state education 
authority. 

I. The project staff selected twenty major basic mathematics 

texts, with their teacher's guides, for analysis and prep- 
aration of system documents — an analysis sheet, the pupil 
text page, and the teacher manual page for each page in 
each of the twenty texts. 

II. a. Specialists in mathematics education developed 

a Content Authority List of more than three 
hundred content items, classified under seven 
general mathematics topics. Content analysts, 
professors of mathematics at colleges and uni- 
versities in the state who were trained for this 
task by the senior editor of the project, as- 
signed codes to each lesson in each book on the 
basis of this analysis tool. They also indicated 
grade level — school year and month — for which 
each lesson is intended. 

B. On the basis of a Behavior Authority List, which 

consisted of fifteen hundred pupil objectives 
developed from the results of mathematics cur- 

ERIC 



riculum research at a university in the state, 
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seven classroom teachers, trained and super- 
vised by the authors of the list, assigned a 
code to every lesson in every text. The 
Behavior Authority List is logically related 
to the Content Authority List; it concerns 
eleven mathematics topics in five major levels 
of skill-activity difficulty. 

C. Two classroom teachers, trained and super- 
vised by the project staff, assigned problem- 
type codes to each page of every teacher's 
manual, identified technical vocabulary and 
symbolism in each series, preparing an alpha- 
betized vocabulary-symbolism index for the 
series, and indicated pre- and post-text in- 
structional activities in each series. 

D. A description of the salient features of each 
book was prepared by the project staff, checked 
with the publishers, and filed at the end of the 
system documents for the text. 

III. A. Coded information is stored in a computer in the 

state's Bureau of Educational Data Processing. 
Authority lists also are stored in the computer, 
where they can be easily updated if need be. 
Indices and other search information can also 
easily be generated by computer. 
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B • Computer-printed analysis pages, pupil text 

pages, and teacher manual pages were micro- 
filmed for permanent storage and to prepare 
aperture cards. 

IV. At the state education authority headquarters and at 

(so far) two regional centers, school districts in the 
state may receive "general," "modified," and "in-depth" 
services ranging from answering queries about approaches 
and features in mathematics texts to assisting in a com- 
prehensive study of the school's current mathematics cur 
riculum, preparation of a new curriculum guide, and se- 
lecting appropriate materials to implement the newly 
designed curriculum. In addition to the complete analys< 
and files, users of the system, whether at headquarters 
or in the regional centers, have the benefit of the ad- 
vice of a mathematics curriculum adviser. 
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Index No. 010: A State Adopts Books at the Elementary Level 

Committees in school districts all over the state 
assist the State Curriculum Commission in preliminary 
evaluation of textbooks, which are adopted on a four- 
to eight-year rotating schedule. 

I. The State Board of Education determines the grade or 

grades in which textbooks will be adopted and appoints 
a State Curriculum Commission to recommend books for 
adoption. 

II. Publishers are informed of adoption plans and asked to 

submit for examination sufficient quantities of texts 
which conform to criteria established by the State Cur- 
riculum Commission. The Curriculum Commission members 
and consultants review the books submitted and select 
ones to be given a more extensive evaluation. 

III. The Curriculum Commission appoints advisers in school 

districts throughout the state to direct the preliminary 
evaluation of books selected for further evaluation. 
Advisers appoint selection committees of classroom 
teachers and school administrators representing all grades 
in all elementary schools in a school district. They 
evaluate books in terms of criteria and rating sheets 
supplied by the Curriculum Commission. At least five 
members of a selection committee review each book and, 
after group discussion, a summary rating report (on forms 
provided) is sent to the Commission. 
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On the basis of the information obtained from the reviews, 
the Commission makes recommendations to the State Board of 
Education. The State Board adopts one or, in some cases, 
two or three basic books (for students at different levels 
of competence) for each subject and grade, as well as 
lists of supplementary texts for most subjects. 
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Index No. 028: Textbook Selection in a Large City in a State 

(010) which Adopts Elementary Texts 

Large representative committees of teachers are re- 
sponsible for selecting basic and supplemental books 
and periodicals for classroom use. 

I. Secondary Level 

A. Books 

1. Subject-matter committees represent- 
ing each school meet to establish 
criteria for selection of basic text- 
books and workbooks and supplementary 
texts. 

2. Publishers are asked to submit books 
which meet the criteria and to discuss 
their books with committee members. 

3. Committee members examine all mate- 
rials submitted and try them in their 
classrooms. 

4. At general committee meetings unsuit- 
able texts are eliminated. Remaining 
ones receive more intensive trial and 
examination. A final selection of 
basic books for a subject and grade 
usually consists of three books (and 
their accompanying workbooks or 
manuals), for students with above- 
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average, average, and below-average 
reading ability. Other books may be 
recommended as supplemental texts. 

5. Selected books are recommended to the 

Board of Education for adoption. 

B. Periodicals 

To add a periodical to the list of "Classroom 
Sets of Periodicals," which are considered sup- 
plementary text material to ensure currency of 
information, at least two teachers in a school 
must sign a recommendation, to be approved by 
the Coordinator, Curriculum, and then submitted 
to the Board of Education, through the Bureau 
of Libraries and Textbooks, for adoption. 
Elementary Level 
A. Books 

1. Basic textbooks are adopted (with the 
assistance of evaluating groups all 
over the state) and supplied by the 
State Board of Education. 

2. The committee which serves as a state 
evaluating group, consisting of about 
thirty-five elementary school teachers 
and administrators, representing every 
geographic location in the city, also 
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selects supplementary textbooks and 
workbooks for use in the schools of 
the city. 

a. Publishers deposit with the 

Teachers Professional Library 
r copies of proposed supple- 

mentary texts. 

b« The staff of the library sees 

that each book is reviewed by 
at least two members of the 
committee, who also try the 
book in several classrooms. 

c. If the two reporting teachers 
disagree, a third teacher is 
asked to review and try the 
book. 

d. The committee discusses all re- 
views before making a recommen- 
dation to the Board of Educa- 
tion. 

B. Magazines 

1. Suggestions for additions to the 

"Classroom Sets of Magazines" iist 
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are submitted by teachers, adminis- 
trators, and librarians to the same 
committee which selects books. 

2. . . Sample issues are examined and evalu- 

ated by at least five members of the 
committee, and, after group discussion, 
final recommendation is made to the 
Board of Education. 













mm. 



mammmm. 






as 



31 

Index No. 019; An Adoption State Channels Evaluations from 

Schools to a Small Selection Committee 

A committee collects evaluations from teachers and 
supervisors and recommends to the State Board of 
Education a number of books to be adopted in each 
subject, on a two- to six -year rotating schedule. 

I. The State Board of Education appoints a four-man Text- 

book and Curriculum Committee to be responsible for se- 
lection of textbooks. 

II. The Committee, with the assistance of the staff of the 

State Board of Education, develops criteria for evalu- 
ating books in areas to be considered in a given year. 

Books are adopted for six years, but in certain subject 
areas one-third of the books on the multiple list are 
adopted every two years. 

III. The Committee appoints persons to evaluate books in each 

area, selecting them from the supervisory and teaching 
personnel across the state. 

IV. Formal hearings are scheduled at which publisher's repre- 

sentatives may make presentations on books being submitted 
for consideration before each Committee member and each 
group of persons selected to evaluate texts. 

V. Groups of persons who evaluate texts summarize their 

evaluations for the Committee which, on the basis of those 
summaries and its own evaluations, recommends a multiple 
list of books in each subject under consideration to the 
State Board of Education. 
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VI. The State Board adopts a multiple list of books and 

notifies publishers and Division (local) Superintendents 
of its action. 
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Index No. 055: A County School Division in an Adoption 

State (019) Selects Books for Local Use 

Committee members decide which of the books on the 
state's multiple list should be adopted for local 
use . 

I. A committee consisting of teachers representing several 

schools in the division, citizens from the community, and 
members of the staff of the division considers each of the 
books adopted for use in schools throughout the state for 
a given area, using specified selection criteria. 

II- The evaluations are summarized and the recommendations 

made to the local School Board to adopt one or more of the 
books on the state list, and occasionally supplemental 
books not included on the state list, for use in the 
schools in the division. 

III. Books adopted by the School Board become the official books 
for six years. 
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index No. 041: A Regulatory Nonadoption State 

A state which leaves evaluation and selection to local 

districts unifies the process by means of central regu- 
lations . 

I. State law requires: 

A. The State Board of Education to prescribe rules 
and regulations governing the choice of text- 
books for use in the public schools of the state 

B. Schools to report their textbook selections to 
the State Board for approval and inclusion on 
an open list 

C. The State Board to negotiate with publishers 
statewide prices for listed books 

D. That an approved book remain on the list for at 



II. 
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least four years 

The State Board, in carrying out the law: 

A. Prescribes criteria against which textbooks 
must be evaluated. * 

B. Defines a textbook as "any book or other mate- 
rial, including but not limited to, printed 
books, recordings, video presentations or films, 
to be used as the basic reference for the im- 
plementation of a course and that alone or in 
combination with other materials will be used 
by all of the students in the course " 



o 



si mm 



5SSSSS® 












35 

C. Makes recommendations concerning evaluating 
primary sources and controversial materials 
before introducing them into courses 

D. Recommends that local districts add to state- 
supplied criteria, perhaps by means of discus- 
sion in advisory committees; depend heavily 
upon the judgment of classroom teachers; con- 
sider in-service training for teachers about to 
use a new textbook; and consider pilot use of 
newly recommended materials, particularly when 
they have been selected under the "controversial 
material" criteria 
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Index No. 096: A School District in a Regulatory Nonadoption 

State (041) Selects Textbooks 

Three types of evaluation for three different situations. 

I. Initial Evaluation 

A. Books to be considered are sent by the Office of 
Instructional Supervisors to selected teachers 
who comment on their general impression of the 
book and suggest further action in connection 
with it — further consideration by a textbook 
selection committee, immediate listing as an 
approved supplementary text, or rejection for 
that school district's use. 

B. When further consideration by a committee is 
recommended, the instructional Supervisor con- 
cerned convenes a committee of teachers to 
evaluate the book in detail and make a recom- 
mendation to the Superintendent of Schools. 

C. Final decision is in the hands of the Super- 
intendent of Schools, including making neces- 
sary arrangements for state approval and list- 
ing. 

II. Re-evaluation of Books in Use 

A. A small committee devises an evaluation form 

which goes to each teacher using the book. 
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B. A meeting of representatives from each school 
is held to consider readoption of t' e book. 
Representatives of schools where the book has 
been used bring summaries of classroom teachers 
evaluations . 

C. After discussion of teachers' reactions and 
examination of the book, a recommendation is 
made to readopt the book or to discontinue the 
use of it. 

D. As in initial selection, final decision is in 
the hands of the Superintendent of Schools, 
including any necessary arrangements for state 
approval and listing. 

Evaluation for a Specific Curriculum 

In the course of in-service training sessions for new 
programs (for instance, a four -day session on Family Life 
and Sex Education for more than twenty teachers) , text- 
books and other instructional materials are evaluated for 



use in a specific curriculum. 
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Index No. 001: An Adoption State Calls Upon 

Two Professional Committees 



A three-level process for adopting a multiple listing 
of textbooks for each area, on a six -year rotating 
schedule . 



I . 



A, 



The State Board of Education receives nomina- 



tions from local boards of education and ap- 
points a Courses of Study Committee in two 



sections, an elementary and a secondary section: 



Elementary : 



Secondary : 



2 teachers from 



3 teachers from each 



each grade 



subject-matter field 



3 elementary princi- 



3 secondary princi- 



pals 



pals 



2 elementary super- 



2 secondary super- 



visors 



1 college professor 
of elementary educa- 



tion 



visors 



1 college professor 
of secondary educa- 



tion 



B. 



The Courses of Study Committee conducts con- 



tinuing studies and evaluations of courses 
taught in the state and prepares contents of the 
courses, considering: 



1 . 



Required basic content 



2 . 



Texts used and available 



3. 



Educational objective of the course 
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4. Changing scientific, technological, 
and cultural developments 

5. Established facts of American history, 
tradition, and patriotism 

C. At least every two years, the committee reports 

to the State Superintendent of Education and to 
the members of the State Textbook Committee with 
respect to: 

1. Compulsory minimum content of courses 
of study 

2. Recommended revisions of courses, 
materials, subject content, and treat- 
ment in specific courses and subject 
areas. 

II. A. The State Textbook Committee consists of twenty 

members. The State Board of Education appoints 
eight elementary teachers and eight secondary 
teachers , nominated by local school superint- 
endents, and one state college professor and 
one state university professor, nominated by 
the State Superintendent of Education. The 
Governor appoints two members who have not been 
employed in the field of education. Committee 
members must file an affidavit that they have 
no connection with a publisher or any 
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businessman which could constitute conflict of 
interest , and that they will reveal the findings 
of the committee only to the State Board of 
Education. Committee members are paid an 
honorarium and expenses. 

B * The State Textbook Committee meets as often as 

necessary to evaluate textbooks in light of 
criteria set by the Courses of Study Committee. 
Rating scales and check sheets are developed to 
record evaluations. 

c * The committee recommends at least six textbooks 

on each subject for adoption in each grade, 
unless fewer than six acceptable textbooks are 
available . 
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A. The State Board of Education announces for a 

period of thirty days the proposed adoptions 
end the time and place scheduled for a meeting 
of the State Board to adopt. Adoptions may be 
made only at a public meeting, and the public 
must be allowed a reasonable amount of time to 
be heard concerning any recommended book. 

B - The State Board of Education shall adopt from 

among the textbooks recommended by the State 
Textbook Committee at least four adoptions for 
each course in each grade unless the Textbook 
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Committee has recommended fewer than four. 

The books become the adopted texts for the 
schools of the state for a period of six years. 
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Index No. 135; A School System in a Nonadoption State Substitutes a 

Variety of Media for the Single Textbook 

Cooperatively developed curricula are implemented by 
means of a master collection of several kinds of media. 

I. A plan, started in the early sixties, to make libraries 

available in each elementary school and to give in-service 
training in library-media center use, resulted in teacher 
requests to review curriculum. A pattern was established 
for such review and for implementation of new curricula, 
as follows: 

A. Research committees investigate current curric- 
ulum trends to define basic objectives and to 
work out the scope and sequence of topics to be 
covered . 

B. A larger faculty committee defines the program 
in more detail and makes suggestions for im- 
plementation, grade by grade. 

C. A "pilot school" tries the program for a year 
and it is introduced, with whatever revisions 
seem necessary, to the whole system — or aban- 
doned. A year-long program of workshops every 
school year ensures that all teachers are in- 
formed about and contribute to new curriculum 
development and keep up with suggestions as to 



methods, media 



services, and activities. 
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II. A basic concept in this cooperative curriculum develop- 

ment has been the abandonment of the single textbook for 
a variety of media. 

A. Media for use in implementing the program are 
basically teacher selected. Suggestions may 
be made by committees and by pilot school 
faculties and sessions are held to introduce 
teachers to new media. However, the teacher is 
the final arbiter of what he will use in the 
classroom. 

B. The school district maintains a large collection 
of media, for the most part housed in individual 
school libraries but centrally cataloged to 
facilitate district-wide use, which the teacher 
may use or ask to have augmented as his plans 
require. Materials for the media collection are 
selected in a variety of ways. 

1. In the case of films, it is sometimes 
possible to have district-wide "film 
shows" at which all teachers of a given 
grade and subject may comment on a 
large number of films. 

2. The basic library collections are main- 




tained by library specialists; special 
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library books are selected as are 
other media. 

3. in the course of trial of new programs, 
committee members or pilot school 
faculty may see that certain materials 
are obtained for pilot use. 

4. After curricula have been installed, 
teachers or curriculum specialists 
may request that the Educational 
Development Center obtain materials 
for preview and may try them out in 
the classroom, thus obtaining student 
opinion. All previews are formally 
reported upon on special evaluation 
forms which are kept on file at the 
Educational Development Center. If a 
teacher requests preview of an item on 
which an evaluation file exists, he is 
given the benefit of the previous re- 
views before deciding finally that he 
wishes to preview the item himself. 



SELECTION OF SUPPLEMENTARY MATERIALS 
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The survey brought forth examples of selection processes for 
library books, for audiovisual software such as films and tapes (with most 
attention given to sixteen-millimeter films for classroom use or for 
transmission by television) , and for maps, models, exhibits, pictures, 
and other such display materials. 

It would appear that almost »all these materials are housed in 
central collections, easily accessible to classroom teachers (and stu- 
dents, certainly in the case of books and more and more in the case of 
other materials) but serving many classrooms. It is for media centers, 
state, regional, and local, that most of the selection descriptions re- 
ceived are designed. 

Two professional organizations, the American Association of 
School Librarians and the Department of Audiovisual Instruction of the 
National Education Association, have cooperated to produce (very recent- 
ly) standards for school media systems, including personnel requirements, 
budget requirements, space and equipment requirements 7 scope, including 
relationships among local, regional-.^ and state centers; selection and 
organization suggestions; and other considerations which make the media 
center a real resource for teaching and learning. In this spirit (and 
already based on these standards in some cases) some state education 
authorities supply standards for their schools to follow in establishing 
iR0(3ia centers (including at least broad hints on dealing with questions 
of controversial materials) . 




45 






mmsmm 






46 

Selection of materials for inclusion in these centers is a 
matter of considerable importance. Persons in charge of such centers 
are likely to bring experience and expertise to the problem, and certainly 
to be aware of the publications which list books and other materials con- 
sidered basic for a media collection. Further than that, there are num- 
bers of sources of reviews and listings, in general periodicals, in 
periodicals designed specifically to serve educators, in professional 
journals, in yearbooks and special bibliographies, which assist in selec- 
tion of general additions to collections, or in obtaining materials to 
serve special purposes. There are even "guides to the guides"--listings , 
sometimes with descriptions, of catalogs, bibliographies, and periodicals 
which are useful in the selection of library books and nonprint media 
materials . 

Media centers frequently enlist the aid of classroom teachers 
in final evaluation of materials which are being considered as additions 
to the collection, not only increasing the probability of choosing actu- 
ally useful materials but also giving teachers an opportunity to learn 
more about what is available to them. When materials can be given trial 
use in classrooms, students' opinions can be taken into consideration in 
purchase decisions. This can be arranged more easily when distributors 
and suppliers are able to lend materials to centers long enough to allow 
trial, or when (as has been increasingly the case under federal legisla- 
tion) funds can be earmarked for the purchase of trial materials. 

The previewing and evaluation needed to make final purchase 
decisions make serious demands of time and effort, and the survey 
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brought forth examples of attempts to alleviate the problems by means 
of cooperative evaluation projects, perhaps undertaken by a state educa- 
tion authority, by all the school systems in a unified geographical area, 
or by a librarians association. Sometimes these cooperative evaluations 
take into account curriculum requirements which apply to all members of 
the group. They may, however, be oriented around a topic, whether or not 
an effort is made to select coordinated materials. 

Multimedia catalogs and guides do exist, prepared by state 
education authorities or by professional associations, to assist in selec- 
tion of various materials for teaching a concept or adding a dimension to 
the curriculum — rounding out the teaching on minorities, for instance, 
or correlating humanities information with the science curriculum. Cur- 
riculum guides prepared by state authorities for local school use may 
suggest materials (in addition to textbooks) to implement the curriculum. 
And some authorities, county or local, issue general criteria to use in 
evaluating all instructional materials, with the aim of achieving balance 
and coherence. 

It will be immediately apparent, in the examples that follow, 
that selection processes for supplementary materials tend to be less 
structured than those for textbooks — for one thing, no example occurred 
where actual selection mechanisms were prescribed by law. 
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Index No. 020; A County School System "Polices* 1 its Media Collection 

Teachers receive honoraria and professional-growth 
credit for participating during the summer in a county- 
wide media review. 

Sponsor: A county school system 

Duration: 1964- 

The purpose of the summer review of media is to add new mate- 
rials to the collection in the county's Educational Materials Center and 
to remove materials which are no longer useful. A three-year rotating 
schedule of reviews is followed, with seven or eight curriculum areas a 
year coming up for review. 

The staff of the Educational Materials Center requests review 
materials from suppliers and prepares lists of currently owned materials. 
They invite teachers to apply for membership on appropriate selection 
teams and make up those teams from among the applicants. They schedule 
the team meetings and coordinate the team activities. 

Each team is composed of from five to seven teachers of the 
proper grade level or subject matter who have not been on previous evalua 
tion teams. They work morning or afternoon for about a week at $3.00 an 
hour. At the discretion of their local school boards, they may also 
receive in-service professional-growth credits at the rate of one unit 
for each sixteen hours worked. 

Teams evaluate the materials on the basis of general criteria 
set by the Educational Materials Center and specific criteria which they 
set themselves. They then prepare a recommendation for the County Board 
of Education, indicating new materials to be bought immediately and ones 
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to be bought when funds become available. They also recommend to the 
Educational Materials Center materials which should be discarded as 

I 

I 

obsolete or erroneous and advise on proper cross-indexing , grade-level 
placement, and subject-area listings of materials in or about to be 
I added to the collection. 
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Index No. 015: Film Selection for State Film Libraries 

To avoid waste of time, the state requires films 
submitted for preview be no more than two years 
from copyright date. 

Sponsor: A state education authority 

Duration: Ongoing 

A "preview coordinator," working with subject-field consult- 
ants of the state education authority, selects films from producers 

i 

i lists to be previewed. 

| The sub j ect— field consultants designate preview committees 

from school and college personnel in_ the state. Members view films 
individually? they complete a "State Department of Education Film Evalu 
ation Form" for each one, to be sent to subject-field consultants. 

Media consultants, film-library managers, subject-field con- 
sultants, and the director and the assistant director of the state's 
Division of Curriculum Development make final decisions as to purchase 



of films for the state libraries. 
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Index No. 030: A Media Center Catalogs User Opinions of Software 

Film users must report on each film they use for a 

summary report. 

Sponsor: U. S. Office of Education, under Title III 

of the Elementary and Secondary Education Act 

Duration: Ongoing 

A media center serving twenty^four local school systems with a 
large collection of software makes items available to teachers who will re 
view them as instructed and report their reviews. 

Specific questions to ask before selecting an item for use are 
supplied, as are instructions to test the students before and after the 
item has been used in the classroom, to assess the usefulness of the 
material. The teacher also must complete an "Evaluation Report" and 
return it with the item. 

Information thus received is treated statistically to show the 
ratio of favorable reviews to total reviews, with indication of the sta- 
tistical significance of the ratio and an indication of the curriculum 
areas in which the item earned favorable reviews. These data are com- 
piled in a "Media Evaluation Report" for the guidance of future users of 
the collection, as well as to suggest local school acquisitions, and to 
provide critical review for film manufacturers. 
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Index No « 042; A County-Wide Filmstrip Review 

The county school - librarians association cooperates 
in the review of filmstrips. 

Sponsor: A county school-librarians association 

| Duration: 1967- 

[ 

On the basis of needs expressed by teachers in schools in the 

1 

I county, librarians obtain from producers filmstrips that relate to curric 
ulum needs. Using guidelines cooperatively developed by the librarians 

k 

association, one of the librarians reviews each filmstrip and prepares a 

1 

t 

| statement for the quarterly publication which the association issues to 

[ 

: 

its members and to interested school personnel. 
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Index No. 054: Curriculum-Oriented Selection for a City's 

Audiovisual Libraries 

Audiovisual materials are selected by committees of 
classroom teachers in terms of curriculum needs. 

Sponsor: A city school system 

Duration: Ongoing 

r 

The Assistant Superintendent for instruction appoints Audio- 
visual Education Advisory Committees, selecting experienced teachers 
recognized for skill in using visuals. Each year there shall be some 
members who are carried over from the previous years. 

The committees represent the three levels of elementary educa- 
tion and each subject-matter field in secondary education. Elementary 
education committees have three teachers from each grade level and a 
supervisor as members? secondary education committees have three mem- 
bers who teach the seventh grade, three who teach the eighth grade, and 
one from each high school. 

Once a year committees review the audiovisual materials already 
in the collection for continued relevance to the curriculum and for time- 
liness, and to determine where there are gaps in the coverage of the 
curriculum. They also preview films and make suggestions for additions 
to the collections, completing for each a cooperatively developed film- 
evaluation card. Their recommendations are submitted to curriculum 



coordinators who are responsible for seeing that teaching tools have 
city-wide utilization and who, therefore, make final decisions. 
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Index No. 045; Cooperative Selection for ETV 

A program director has "dissatisfied users" choose 
replacements for films shown in instructional tele- 
vision. 

Sponsor: Member schools and a state education 

authority 

Duration: Ongoing 

Member schools (forty public districts and fifty-five parochial 
schools) send teachers to serve on central committees dealing with seven 
major curriculum areas who review and comment on programs transmitted 
during the past year by a cooperative educational television station. 

Committees whose reactions are favorable are disbanded. Others 
are constituted into selection committees who meet to preview and dis- 
cuss films and video tapes and make recommendations for the coming year's 
television programs. They may recommend use of material they have pre- 
viewed, or give specifications of what they would like to see transmitted. 
In the latter case, the station makes every effort to obtain suitable 
material; if none can be found, they produce materials to the committee's 
specifications . 

The committees for selection meet about eight times each over 
a six -week period. Their schools or systems arrange for and pay substi- 
tutes for their classes, and pay any expenses incurred in connection with 
committee work. 
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Index No. 089: Evaluation of Nonprint Media to Implement a 

New Approach to the Curriculum 

A state prepares guides to and kits of resources 
to relate cultural enrichment materials to courses 
in English , science, social studies, industrial 
arts, and home economics. 

Sponsors: U. S. Office of Education and a state 

education authority. 

Duration: 1962- 

Participating schools submitted descriptions of their courses 
of study to personnel of the state education authority, who studied them 
to see where the arts and the humanities might be integrated into the 
regular subject matter. Preliminary selection among a very large col- 
lection of materials was made in light of those deliberations. Then nine 
teachers selected further among the materials, relating appropriate ones 

to lesson plans which they devised. 

Kits consisting of the lesson plans and the materials selected 
to implement them were placed in the cooperating schools for trial and 
evaluation. Students in participating schools and in control schools 
completed rating scales on works of art and were interviewed as to their 
attitudes toward the arts, and teachers submitted ro the state education 
authority their reactions, evaluations, and recommendations concerning 
materials and lesson plans. 

Offerings were revised in light of the above evaluations to 
arrive at lesson plans, and lists of materials to implement them, which 
could be disseminated throughout the state. 
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index No. 131: introduction of a New Medium 

A media center tries film loops in five member schools. 

Sponsor: U. S. Office of Education, under Title III of 

the Elementary and Secondary Education Act, 
a state education authority, and local 
schools in the region. 

Duration: Unknown 

Film loops were obtained on loan from producers for a semester 
and divided among teachers in five participating schools. Teachers were 
informed of criteria against which they would be asked to evaluate the 
film loops at the end of the semester. These criteria include estimates 
of appropriateness, effect, and quality of the materials, and their use- 
fulness (particularly as compared with filmstrips) in the curriculum. 

Also supplied to the teacher were a rating scale on which he could record 
his own reactions to the use of the loops and his observation of student 
acceptance of them, and a rating scale to be administered to students with 
whom they had used the loops. All evaluations were of the loop system in 
general, not of specific loops, and the information was used by the media 
center as a basis for decision as to whether to add loops to its collec- 
tion. 
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Index No, 092: A City School System Adds to Its Libraries 

j Teachers, parents, and students can suggest titles 

I for the school libraries. 

Sponsor: A city school system 

i Duration: Ongoing 

'i. 

\ Secondary Level: Two committees of librarians, one for junior 

I 

f high schools and one for senior high schools, have final say on books and 
f periodicals for school libraries. Suggestions for books are sought from 

I 

I administrators, teachers, librarians, students, and parents. 

I 

I 

f Each librarian in the system is responsible for the review of 

two books a week, by himself or by teachers and administrators he se- 

I 

| lects. Books normally receive two reviews, though some are reviewed 

I 

| 

only once. Teachers may ask to review particular books. 

£ 

; 

Reviewers prepare written reviews for the Bureau of Library 
j and Textbooks. If reviews are widely discrepant, a third review is 
[ sought. If a review from a teacher who has asked to review a specific 

j book raises a question, a teacher in another school is asked to review 

i the book also. 

Each committee of librarians meets four times a year to con- 
sider the written reviews and to select books to be listed, with annota— 

I 

j 

j tions, on the approved buying list. The committees also enter, review, 

i or cancel subscriptions to periodicals once a year on their own responsi- 

t 

\ 

bility but with the advice of teachers and administrators. 

Elementary Level: The Elementary Schools Book Committee 

(which also deals with textbooks and other classroom aids under state 
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adoption regulations) is responsible for selection of library books and 
periodicals. 

All books submitted by publishers are reviewed, but priority 
is given to those which have been recommended by teachers and adminis- 
trators. Two members of the committee read each book and they are used 
in classes to obtain teachers' and students' evaluations. 

If teachers' evaluations are discrepant, the process is re- 
peated and all results discussed by small groups within the committee 
before the committee meets to make recommendations for the purchase of 
books . 

The committee obtains suggestions for additions to and dele- 
tions from the "approved library magazine list" from teachers, adminis- 
trators, and librarians. Members of the committee examine the magazines 
and, after committee discussion, the list is prepared. 
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Index No. 039: Nonprint Materials for a Specific Topic 

A group of librarians decides to evaluate all media 
available on a given topic, starting with Afro- 
American culture. 

Sponsor: A state education authority 



Duration: 1968- 



More than twenty librarians and audiovisual specialists repre- 
senting public schools, private schools, public libraries, library 



schools, media centers, and the state education authority, organized for 
the purpose of attacking the problem of evaluating media for use with 
children and young people. 

After investigation of various response forms for use in evalua 
ting media, the committee decided upon a "universal" one which permitted 
some variation to accommodate characteristics of individual media but 



which would permit uniform reporting. 

The committee chose Afro-American culture as the first area in 
which media would be evaluated. A subcommittee was selected for each 
type of medium, and chairmen were instructed to obtain materials from 
producers and also to incorporate into their reports pertinent and recent 
reviews which had been developed for a large school system represented ir 
the group or for the State Division of School Libraries. 

The evaluation reports were edited and prepared for printing 
and distribution to school personnel by the members of the committee and 
by the branch in the region of the State Division of School Libraries. 



The report will be ready in late 1969. 
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Index No. 091; A State "Adopts" Library Books and Audiovisual 

Materials for Local Schools 

Schools may use state funds for the purchase of 
library books and audiovisual software only if the 
materials appear on a state -approved list. 

Sponsor: A state education authority 

Duration: Ongoing 

Elementary books: Each member of an Elementary Materials 

Committee of ten members — reading specialists, supervisors, professors 
of education or library science, state board members and officials — 
reads all books submitted by publishers for inclusion on the elementary 
school library list and prepares a confidential written evaluation of 
each one. These evaluations are discussed during a week-long meeting to 
arrive at a consensus as to what books to recommend for the state-ap- 
proved list. 

High School and Adult Books: Two members of the High School 

Materials Committee and one high school librarian read and prepare con- 
fidential written reports on each book submitted for inclusion on the ap- 
proved high school library list. Approximately one hundred local librar- 
ians participate in the selection process. 

The High School Materials Committee, like the Elementary one, 
meets for a week to discuss the evaluations and make recommendations for 
the approved list. 

Professional Books: Members of the staff of the state educa- 

tion authority review professional books and make recommendations for 
the approved list. 
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\ 

[ Audiovisual Materials: Members of the staffs of the state 

l 

l education authority and the state Bureau of Teaching Materials review 

I audiovisual materials for inclusion in the state's free teaching— 

f 

I 

I materials service and on the state-approved list. 
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index No. 022: Media Selection on a Coordinated Curriculum Basis 

City-wide committees are responsible for selection 
of all materials to be added to media centers. 

Sponsor: A local school district 

Duration: Ongoing 

The Superintendent of Schools appoints committees, made up of 
principals, teachers, librarians, and subject-matter or grade-level con- 
sultants, to select all materials to be added to the instructional media 
centers or used with texts in classrooms. The areas of responsibility 
of each committee are carefully detailed. There are two elementary- 
level committees, one to select classroom learning materials and one to 
select library-media center learning materials. Each may be divided into 
subcommittees representing grade levels for certain selection tasks. 

Secondary-level committees represent broad curricular areas; 
they may operate in subcommittees by grade level (English, homemaking, 
mathematics, and social sciences), subtopic (science and music), or sex 
of student (physical education and safety) . For art education, for each 
foreign language, and for business education and industrial arts, all 
teachers serve as the selection committee, with the librarians and the 
consultants. 

The committees are charged with the responsibility for "the 
acquisition of a well-balanced and coordinated set of learning materials 
for use of students and teachers." Attendance at committee and subcom- 
mittee meetings and preview sessions is considered sufficiently important 
to make it necessary to send an alternate if one cannot attend. 
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Index No. 003; A State Supplies Schools with Film Reviews 

To assist audiovisual directors and teachers in 
the state to keep up with new materials as they 
are produced,' a state education authority in- 
stitutes a review system, starting with films. 

Sponsor: A state education authority 

Duration: 1963- 

The state education department staff devised a loosely struc- 
tured response form designed to elicit overall judgments and the reasons 
for them, recommendations and cautions concerning the use of a film, and 
straight descriptive information from reviewers. 

A group of approximately one hundred teachers, representative 
of every section of the state and every type of school organization in 
the state, were selected to evaluate films. They initially met in groups 
in approximately ten locations in the state for instruction and practice. 

Each film received a minimum of three reviews. Reports were 
synthesized to form one statement about each film. These statements 
were printed in a guide distributed to all schools in the state. 

On the basis of initial experiences, it was decided that more 
thorough training of the evaluating teachers was required and that a more 
structured response form was required, one which lent itself to the prep- 
aration of responses for computer storage. 
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EQUIPMENT SELECTION 



Equipment selection often requires two decisions — whether a 
school or district should introduce a new type of equipment and, if so, 
what make and model to buy. Such considerations, of course, vary con- 
siderably in complexity, and it is likely that for major installations 
advice and assistance must be sought from consultants— —perhaps engineers. 
There are, however, some manuals to assist in at least the preliminary 
decisions about major equipment. 

For simpler installations, after considerations such as avail- 
ability of suitable useful "software" and need for special skills to 
operate the equipment have been investigated and satisfactorily resolved, 
the equipment itself comes under scrutiny. When that occurs, unfortu- 
nately expertise and physical facilities needed to make comparative 
tests of equipment are usually lacking. Thus much audiovisual equipment 
selection consists of so-called "hands on" evaluation — the potential user 
tries one or two models and buys the one he likes. He may consult a 
colleague or a published selection aid. 

Some information is available from professional and trade or- 
ganizations on various types of equipment, and the EPIE Institute in- 
cludes comparative descriptions and evaluations of audiovisual equipment 
in its publications and services. Other aids, based on more-or-less 
extensive evaluation, may be made available by central agencies— —a state 
or a large city education authority, a testing facility established by 
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an organization whose members make extensive use of equipment, a com- 
mercial testing laboratory, or an agency with incidental but extensive 
educational responsibilities. (One of the most succinct and useful 
equipment evaluation reports to come up in the survey is issued by the 
central office of a Protestant church.) These aids, based on evaluation 
undertaken for the purpose, typically take the form of criteria against 
which to compare equipment, or comparative catalogs — though one very 
interesting one is a series of tests for nonelectronically oriented 
language teachers to make on — and with — language laboratories before 
paying for them. Some states set fairly detailed standard^ concerning 
such factors as performance, convenience, and durability, below which 
equipment may not fall if it is to be bought with state or state-admin- 
istered federal funds. Such standards are useful also as guidelines 
when local funds will be used, of course? they and others designed only 
as guidelines are sometimes compilations of reports from typical users 
who have given the equipment a trial use or have participated in a work- 
shop on it. In at least one workshop, described in some detail later, 
the participants used their newly developed criteria to prepare a com- 
parative catalog. 

Such comparative catalogs are more often developed on the basis 
of laboratory testing, though field test results may be included. When a 
large enough population is available for survey, catalogs may be sum- 
maries of reports of users. Some catalogs merely give the results of 
identical tests for each item of equipment, leaving all value judgments 
to the catalog user. Others venture to advise and compare, particularly 
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if designed primarily for use within the administrative circle which 
caused the catalog to be prepared. 

It should be mentioned that some guidelines are prepared by 
experts on the basis of experience and knowledge rather than as a result 
of formal tests and surveys. An example could be cited of a set of 
standards and guidelines available in one state to schools which may want 
to supplement television instruction. The guidelines define the two major 
supplemental systems and compare then as to educational needs to be sup- 
plied, technical considerations, and cost factors. Instructions for 
making the necessary legal arrangements for the more independent of the 
two systems are given in detail. The guide was prepared by persons in 
the state who are experienced in the use of all types of educational 
media. 

Finally, it should be mentioned that economics and feasibility, 
always and properly influential in the selection process, sometimes must 
become overriding considerations. A case in point is a guide to video- 
tape recorders issued by one state department of education to schools 
which expect to use the state's videotape services. Since so far there 
is no compatibility among the many available videotape systems, the state 
felt constrained to advise schools to buy only the brand of equipment 
which the state uses. 

The examples which follow describe in more detail how some of 
the evaluation aids were developed, how they are sometimes supplemented, 
and how institutions without access to such aids evaluate equipment. It 
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Index No. 086: A Statewide Committee Looks at Equipment 

A special state committee examines and tries out 
equipment for information of all state agencies. 

Sponsor: A state purchase authority 

Duration: Ongoing 

A committee chosen from faculties of state and community col- 
leges and the state university, and from the state's budget and standards 
committees, is charged with the responsibility of appraising equipment on 
behalf of all state agencies; resultant information is available on re- 
quest to local school districts and to private schools. 

All equipment is tried out in the field. Frequently the com- 
mittee meets with producers to discuss and try out equipment and then 
places the equipment in the field. Reports on trials are discussed at a 
committee meeting and recommendations are made to the state purchasing 
authority. 
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index No. 056: A Field Check Manual 

A state education authority assists teachers in 
the evaluation of language laboratories. 

Sponsor: A state education authority 

Duration: 1964- 

On the basis that carefully selected and carefully manufactured 
and installed equipment may require adjustment before it operates at its 
best, a state education authority has devised a "F ; eld Check Manual for 
Language Laboratories: A Series of Tests Which a Non-Technical Person 

Can Conduct to Verify Specifications." It is recommended that the field 
check be made prior to final payment for the laboratory to determine 
whether further testing by a qualified technician is required. 

The manual contains a series of simple tests, to be conducted by 
a teacher alone or with the cooperation of students. The teacher must 
operate the laboratory and look or listen for certain effects. To check 
frequency response, he must read a list of one-syllable words through the 
equipment to students who are to record exactly what they hear. He must 
answer questions about "downtime" and maintenance service. He is advised 
to compare specifically the specifications in the order for the laboratory 
with the equipment installed. 

Information for interpreting most of the tests is contained in 
the manual itself; unusual results may have to be referred to state au- 
thorities. The tests were devised by a consultant from the State Bureau 
of Engineering and the State Supervisor of Modern Foreign Languages, with 
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assistance from a bulletin on language laboratory facilities from the 
U. S. Office of Education. 
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Index No. 037; Selection in a University 

The Educational Media Services at a university 

select audiovisual equipment for purchase. 

Sponsor: A university affiliated with a religious 

denomination 

Duration: Ongoing 

Staff members of the university Communications Services Div- 
ision (of which Educational Media Services are a part) apply their par- 
ticular expertise to the scrutiny of equipment under consideration. Each 
person who examines a piece of equipment records quantitative data, 
checked for accuracy, on an "Electronic Media Evaluation Report," which 
requires information on technical characteristics, expected maintenance 
requirements, ease of operation, and general engineering quality. Deci- 
sions to purchase or not are made on the basis of these data. 
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index No, 052; Selection in a City System 

A large city school system subjects audiovisual 
equipment to tests and field trials. 

Sponsor: A city board of education 

Duration: Ongoing 

Equipment is obtained and subjected to tests of electrical, 
mechanical, and thermal operation, construction features, and safety 
factors. Convenience in instructional situations and usefulness for 
various specific school needs are judged, as are maintenance factors — 
accessibility of mechanism, availability of parts, need for special tools, 
etc. Equipment which warrants further investigation is sent to schools 
for field tests. A committee makes a report, on the basis of results of 
the two types of tests, to the Bureau of Supplies. 
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Index No. 128; A Dual-Purpose Workshop 



A state arranges for guidelines for selecting equip- 
ment and appraisals of locally available models. 



Sponsor: Audiovisual center of a state education 

authority 



Duration: 1966-1967 



A team of fourteen teachers, librarians, and audiovisual spe- 



cialists (or persons who assumed two of those roles) met for two weeks in 



an Audiovisual Workshop to establish criteria for evaluating projectors 



of various kinds, record players, tape recorders, and projection screens 



Local distributors made the equipment available and demonstrated its use 



The participants then worked with, examined, took apart, reassembled, 



and otherwise handled the equipment in order not only to appraise it 



but also to devise a form for use in appraising any audiovisual equip- 



ment . 



The two goals were met, with a resultant report that includes 



criteria sheets for use in evaluating audiovisual equipment and results 



of such evaluations for locally available equipment. 
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Index No. Oils Laboratory Tests of Videotape Recorders 

A state contracts with an electronics laboratory 
to test thirteen videotape recorders to assist 
purchasers in school systems. 

Sponsor: A state education authority 

Duration: 1966 

At an electronics testing laboratory equipped for just such 
tests , a team of four electronics specialists conducted a series of tests 
on thirteen models of videotape recorders obtained from seven producers. 
The tests are described in detail in a report, and pictures are included 
for each recorder at the same stages in the tests. Also included are 
mechanical comparisons and a chart of specifications as supplied by the 
producers. The facts are presented without editorial comment. 

The report also includes information on lighting and on visual 
aids to improve instruction by television, on the availability of in- 
structional television materials, including a directory of suppliers, 
and on the cost of kinescope recordings. 
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Index No. 094: An Evaluation to Assist Purchasers 

A church-related agency looks at Super-8 projection 
equipment and films to assist churches and schools 
in their purchases. 

Sponsor: A large Protestant denomination 

Duration: 1967-1968 

On the basis of criteria set by a church-appointed committee, 
the agency staff selected four Super-8 projectors for intensive evalua- 
tion and obtained samples from producers. 

An independent laboratory made test film to the agency's speci- 
fications for each of the projectors; each one was run hundreds of times 
and then rated for wear. In one case where wear was excessive, the 
producer was notified and upon investigation made arrangements to re- 
design or rework the cartridge and head. The agency then arranged for 
purchase of corrected equipment from a dealer unaware of the nature of 
the sale and repeated its tests. 

Other electrical and mechanical tests were applied and equip- 
ment was shipped by express to test sturdiness. On the basis of agency 
recommendations, a model which had been damaged in one shipment was 
altered by its producer and the new version stood up well to fifteen 
days of express shipment. 

Projectors were also used in the field before evaluations were 
written, and the original evaluations were updated as additional informa- 
tion or revised models became available. Results are reported in a 
fashion which leaves little doubt of the reporter's preference. 
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The report also includes a description of Super-8 film and a 
comparison of magnetic and optical sound. It also contains price lists 
for having Super-8 film printed at two commercial film laboratories. 
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I index No, 004; A Survey of Equipment in Use 

| A state education authority surveys every school 

f district in the state to obtain information on 

educational communication practices and needs. 

| 

\ Sponsor: A state education authority 

I 

j Duration: 1965-1966 

r 

| A detailed questionnaire was sent to each school district in 

the state, to be completed and returned "by the person responsible for 

| educational communications (audiovisual, television and electronic class— 

I 

| rooms, programmed instruction) for [the] school district." The first 

| # 

| part of the questionnaire inquires about personnel complement, machine 

I 

! complement, budget, and ability to fulfill need for services? the rate 

I 

| 

I at which use of services is increasing and the problems with supplying 

I services come next. The third section deals with software availability 

I 

in the district and the fourth solicits opinions of brands of equipment 

1 

1 

j in use within the district and an estimate of the frequency of repairs. 

After requests for description and rating of any language laboratory in 

I 

use in the district, and for information about the use of television 



there, the questionnaire inquires about the relationship between the 
educational communications program and the school library program in the 
district and requests a rating of the need for certain services which can 



be provided by the state education authority. 

Summarization of the responses, for instance, on the mechanical 
aspects of the overhead projectors in use around the state gives a 
valuable catalog to assist in evaluation. For each projector in use are 
listed the school districts which use it, how many each district has, 
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the rating (A to F; A = best, F = poorest) of the district director of 
Educational Communications as to its mechanical characteristics, and a 
rating (on the same scale, A to F) of its acceptability for use by 



teachers. 






SPECIAL-PURPOSE SELECTION 



Special Education Materials 

In recent years, special education has received increased at- 
tention as increased possibilities for diversity in instruction have 
made the tailoring of educational procedures to special conditions some- 
thing which could be considered. Selection of materials for other than 
typical students is not necessarily accomplished differently from any 
other selection, except that a special expertise is required of the 
persons making the judgments, and there is an emphasis on trial use and 
post-use evaluation. 

It is of interest to note that each of the projects described 
herewith is funded by the U. S. Office of Education. There is no reason 
to believe that some other education authority, state or local, might not 
fund such investigations, but no examples came forth in the course of 
the survey. 
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Index No. 040; Selection for the Mentally Retarded 

A state education authority undertakes a program to 
identify, evaluate, and establish criteria for the 
selection of instructional material for use with the 
educable mentally retarded. 

Sponsor: U. S. Office of Education, under Title VI -A 

of the Elementary and Secondary Education Act 

Duration: 1968-1971 

A steering committee of six — the project director from the state 
education office and a regional coordinator from each of the five (out of 
twenty in the state) educational service centers concerned in the project — 
meets four times a year for week-long periods with representatives of 
publishing companies, who are asked to correlate available materials to 
the curriculum for the mentally retarded. The steering committee deter- 
mines which materials shall be subjected to intensive study. These 
materials are purchased and placed in classrooms — one in each of the five 
regions — for trial use. In-service training is given where needed. 
Evaluations are used in determining what available material can be used 
with mental retardates and how it should be sequenced, and to indicate 
areas where materials must be developed. 

Areas for which materials are considered are ones to develop 
personal and social competences, physical competences, and vocational 
proficiencies, at each level of the curriculum (elementary and second- 
ary) for the mentally retarded. 
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Index Nos. 057 and 058; Selection for Migrant Children 

Two subsidiary projects supply evaluative informa- 
tion for a statewide project for education of 
migrant children. 

Sponsor: U. S. Office of Education, under Titles I 
and III of the Elementary and Secondary 
Education Act 

Duration: 1967-1968 

Project One: Prior to the opening of a demonstration school 

for migratory children, a five-member committee (first grade, fifth grade, 
and special— education teachers, an elementary supervisor, and a counselor) 
reviewed existing materials which might have relevance for migratory 
children, indicating probable usefulness and suggested grade level of 
ones to be subjected to further investigation. These materials were tried 
in the demonstration school and the results of the trial were incorporated 
with the committee's comments in listing materials as an aid to teachers 

of migratory children throughout the state. 

Project Two: A regional education service center, in order to 

provide sixteen-millimeter films suitable for use with migratory students 

in kindergarten through grade nine, had thirty teachers of such children 

preview and evaluate films suggested for the purpose by their producers. 

/■ 

Materials were purchased on the basis of these evaluations. A catalog of 
the films is supplied to schools, which may borrow the films from the 
service center. Users are supplied with a form to supply the center with 
evaluations based on usage. At the end of the school year the collection 



is 



revised on the basis of circulation figures and the evaluations re- 



turned with films 
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Index No. 093: Selection for the Handicapped 

A Special Education Instructional Materials Center 
selects and makes available for examination materials 
usable with handicapped students. 



Sponsor: U. S. Office of Education, under Title VI -A 

of the Elementary and Secondary Education Act 



Duration: 1968- 



On the basis of catalog listings, recommendations of teachers, 
or other descriptions, the center obtains materials for evaluation. Most 



of the materials are purchased through grants. Some, however, are given 
to the center by producers. The materials are used in classes for special 
students (hard of hearing, partially sighted, mentally retarded, percep- 
tually handicapped, or emotionally disturbed). Teachers submit written 
reports, which are kept on file at the center. In some cases, on the 
basis of the teachers' reports materials are subjected to more intensive 
evaluation. 

These materials are kept at the center for examination. Work- 
shops of five or six teachers at a time may discuss the materials with 
representatives of producers, and materials are circulated for trial 
use by teachers. Workshop participants and trial teachers submit written 
reports, which are used to prepare recommendations to producers and ad- 



vise on purchase for schools. 
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j index No. 140: A Special Education Instructional Materials Center 

[ Specializes in Cooperating with Producers 

1 

P 

P 

f This center, one of the few located within a state 

I education department, has access to a large number 

| of classrooms. 

■ 

I 

j Sponsor: U. S. Office of Education, under Title III 

| of Public Law 89-105 

% 

t 



J Duration: 1967- 

l Materials for teaching mentally, physically, and emotionally 

k 

i; 

i 

j impaired children are bought as they become available and distributed for 

f 

| trial use in as many classrooms as possible, through a network of three 

| regional and thirty— one associate centers. Teachers report to the Center 

| 

their reactions to the materials in use, and those materials receiving 

s 

'I favorable reviews are described and recommended in SEIMC publications. 

| Suggestions for improvements are discussed with the producers of the 

| 

| materials. In line with that effort to cooperate with producers, the 

is 

I Center has served producers as a developmental trial center and has re— 

i 

l 

jj viewed and made recommendations on producers' collections. 
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Index No. 129: Selection for the Deaf 

A national professional organization devises stand- 
ards for library-media centers in schools for the 
deaf. 

: 

Sponsor: Office of Captioned Films for the Deaf, 

U. S. Office of Education 

i 

I 

! 

| Duration: 1966-1968 

| 

| After a status study of libraries in schools for the deaf, 

committees of school library specialists and of teachers, librarians, 
and administrators in schools for the deaf, using the American Library 

*>: 

| Association's Standards for School Library Programs, developed a compre- 
[ hensive set of guidelines to assist in the establishment and improvement 

I 

of media centers in schools for the deaf. Major areas covered are pro— 

I 

1 

[ gram, including services to individuals and groups; personnel, including 

a 

3 

l 

special qualifications and professional training required of the staff, 
as well as quantitative standards for staff; collections of printed and 
j of audiovisual materials, again including quantitative standards; quarters 
and equipment; and annual expenditure required for proper maintenance of 



the collections. 
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Programed Instruction Materials 

The selection of programed instructional materials is somewhat 
different from other materials selection since the material itself is 
different in nature from other teaching materials. The survey brought 
forth only one example of a plan for evaluating programed material? it 
is described on the next page. 

For assistance in selecting programed materials there are sev- 
eral publications. Guidelines for use of producers and users of pro- 
gramed instruction materials have been set down by a joint committee of 
three professional organizations, the American Educational Research 
Association, the American Psychological Association, and the Department 
of Audiovisual Instruction of the National Education Association. The 
U. S. Office of Education has issued guides to programs available. They 
describe the programs in some detail, giving sample frames as well as 
information as to stated purposes, availability, and cost. A book on 
self-instructional programs published in 1966 gives detailed instructions 
for evaluating a program, with a check list to assist in doing so. The 
EPIE Institute has considered programed instruction materials for sev- 
eral grades and subjects. 
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index No. 095; Selection of Programed instruction Materials 

A branch of the military examines programed instruction 
materials for use in its training programs. 

Sponsor: A branch of the United States armed forces 

Duration: Ongoing 

The selection is carried out by an instructor, who first com- 
pares the objectives of the materials with the objectives of the course. 
He then examines the text for suitability to those objectives and the 

validation data for evidence of success. 

A comparison is made of the characteristics of the validation 
population and the student population. Significant differences may not 
rule out the programed material under consideration but may make it nec- 
essary to make adaptations, such as a change in terminology in order to 
adapt a civilian program to military use, or provision for additional 
training for the student population before they attempt the program. 

The program, as adapted if need be, is then tried on a repre- 
sentative sample of the student population. This branch of the service 
requires that ninety percent of the students achieve the objectives of 

programs used in its training procedures. 

Then must be considered, before a final decision is made, 
the questions of increased effectiveness, savings in teacher and student 
time, cost factors, and availability of qualified instructors and equip- 
ment. Material is adopted only if its use will result in training im- 
provement . 
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Standardized Tests 



References to selection of standardized tests were almost en 



tirely absent from this survey, occurring peripherally in connection with 



textbook programs which sometimes include tests and in a set of guidelines 



for overall evaluation of special-purpose education programs, which con- 



tains a succinct statement of points to be considered in evaluating a 



test . 



Test publishers do sometimes make available guidelines and in- 



structions for matching tests to instructional plans and to students. 



There have been, too, guides on selection of tests for specific purposes, 
notably, a detailed comparison published by the International Reading 



Association of five tests of reading readiness and five elementary read- 



ing achievement tests. The most comprehensive selection aid, however, 



is a massive and often up-dated compilation of reviews of all standard 



,j t 

ized tests by test makers and other evaluation specialists. 
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Special Curriculum Materials 

Another aspect of materials evaluation mentioned in the course 
of the survey is selection of materials to fit into a specific program. 
Three examples are adduced here, two designed for the individualization 
of instruction and one to bring about change in the teaching of social 
studies . 
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Index No. 104: Selection for Individualized instruction 

An innovative junior high school selects materials 
for inclusion in learning concept packets. 

Sponsor: A city school district 

Duration: 1966- 

Students progress through courses at their own speed as they 
complete "Unipacs" — single -concept packets of materials exploring the 
concept, the objectives of instruction in it, and the technique to be 
used in teaching it, and containing lists of learning materials needed 
and copies of evaluation instruments. Teaching teams, composed of 
teachers, teacher aides, paraprofessionals, and perhaps student teachers 
select materials to be included in Unipacs on the basis of their knowl- 
edge of objectives to be fulfilled. The team meets again after the 
majority of students have completed a Unipac to revise it, if need be, 
on the basis of the performance of the students on tests of their master 
ing of its objectives. 
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Index No. 

A staff 
to meet 



105: A Social Studies Project 

of social studies specialists helps 
their social studies teaching goals. 



schools 



| Sponsor: U. S. Office of Education, under Title III 

I of the Elementary and Secondary Education Act 

I 

I 

| Duration: 1968- 

! 

[ This social studies project offers its entire package of serv- 
ices, including an in-service retraining program now in a developmental 



stage, to any public, private, or parochial school in the area it serves, 
to help them implement an innovative social studies program. The staff 
of the project sets forth a social studies curriculum recommendation 
(based on the latest research findings in social studies education) for 
kindergarten through twelfth grade. Cooperating schools can make a 
diagnosis of their social studies programs on the basis of these recom- 
mendations. As part of its research, the project asks students and 
teachers for paired comparisons of materials in innovative social studies 
programs (developed elsewhere) in the light of the project staff' s recom- 
mendations. Materials evaluations are to reflect the recommendations; 



include descriptive, analytic, and evaluative elements and show the ef- 



f 



feet of the teacher on the success of the course. 



index No. 106: Available Materials in Individualized Instruction 



An experiment in compute r-managed individualized 
education. 

Sponsors: A private education corporation, a non- 

profit behavioral-science research 
institute and twelve school districts. 

Duration: 1966- 

Twelve school districts across the country are cooperating in 
an experiment to individualize education, using currently available in- 
structional materials in conjunction with carefully specified educational 
objectives and criterion-referenced achievement monitoring. A computer 
is used to facilitate recording of progress and matching of materials to 
students. Participating teachers and subject-matter specialists construct 
teaching-learning units which guide the students through instructional 
activities chosen to maximize pupil attainment of sets of specified ob- 
jectives; these units are revised on the basis of student performance 



on post-unit tests. 
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APPENDIX B 



A NOTE ON EVALUATION FORMS 

The survey brought to the attention of the compilers so many 
questionnaires , check lists, and rating sheets used in the process of 
selecting instructional materials that it seemed worthwhile to comment on 
these examples in particular and on the use of such instruments in gener- 
al . 

On several important points in the use of the evaluation in- 
struments received, evidence is lacking. If manuals or instruction pro- 
grams for the completion of the forms exist, no examples accompanied the 
forms submitted in the survey. Similarly, no plans for monitoring the 
evaluations, or for synthesizing and summarizing responses, or for inter- 
preting, using, or disseminating results, were received, though through- 
out the implication is that purchase decisions rest on the results ob- 
tained from such forms, and in some cases it was stated that the forms 
aire filed, so as to be available to anyone interested in the material 
evaluated . 

The mere existence of the nearly fifty forms received in the 
survey reflects the continuing quest for objectivity in education. That 
does not mean that it would be possible to characterize all the forms as 
"objective," and it is likely that the most "objective" ones are the 
least satisfactory for obtaining a good base for making selections. The 
question arises as to whether objectivity in this connection should be 
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prized above all else. The user's subjective judgments of its rationale 
and principles are of utmost importance in his approach to and use of a 
text, or a film, or a tape. 

And subjective opinions are not made objective by use of a 
rating scale, though carefully worded questions in a rating scale can 
sometimes channel subjective judgments in a useful way. Nevertheless, 
even such forms, if they permit or request responders' comments, are 
apt to achieve more pertinent evaluations than rating scales alone. 

This is not to say that the very useful characteristics of 
objective evaluations must be done away with in the selection process. 
Quantification of check list and rating-scale responses certainly is 
possible, and it would serve a useful purpose in screening, for instance, 
so that only those subjective comments which deal with potentially use- 
ful materials need be considered. However, such quantification should 
be based on research to determine the relative importance of items in 
the rating scale. One form received did indeed indicate differential 
weightings of responses, but information on how weights were assigned 
was not supplied. Another form used a numerical rating scale and as- 
signed a simple total as a "score" for each book being considered. Such 
diverse elements as "quality of typography" and "adequate coverage of the 
subject matter" are given exactly equal weight in those "scores," which 
thus become dangerous as a basis for selection. 

Very few of the forms received fall into such traps of misused 
scientism; most of them were designed to elicit information to be used 
qualitatively, and as such, undoubtedly, added a great deal to the 




U 

1 



3 



5 



I 

5 

I 

* 



i 







i 



| 

A 

ii 

X 

h 

% 

■J 

j 



i 

I 

j 

I 



a 




4 






O 

ERIC 












wmm 



mm 



\ — 



96 

selection process. Nevertheless, they did not avoid all pitfalls, and 

they did miss some opportunities. 

In some cases, the forms indicate a search for a universal 
rating sheet, check list, or questionnaire for product evaluation. No 
such form is possible , and to search for it is to avoid the problem of 
developing useful, reliable, valid instruments for the job at hand. 
Specific forms, directed at the points considered important by the group 
responsible for selection, provide the most useful basis for decision 
making. 

Implicit in the forms received is the notion that the person 
completing the form is thoroughly cognizant of all aspects of the cur- 
riculum in which materials will be used and of all characteristics of the 
students with which, and the teachers by whom, they will be used. Further 
than that, there is a tacit assumption that the curriculum is the same 
for all classes (which it may be) , that the student groups are all homo- 
geneous and exactly alike (very unlikely) , and that the teachers are 
identical in skill and training (also very unlikely) . In only the rarest 
case is there opportunity to indicate that a material is suitable for 
some students and not for others, and at that, the only distinction among 
students is their reading level: "above average," "average," "below 
average." Nothing about ethnic groups, student interests, or socio- 
economic levels appears in any of the forms received. And there is no 
way to indicate that one book may be of particular help to an inexperi- 
enced or not very well-prepared teacher and that another can serve well 
only if the teacher brings certain skills and training to the job. 
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Only one of the forms received specifically called for answers 
from students, and none mentioned parents or administrators. Overwhelm- 
ingly, the teacher is the respondent. It seems logical that he be central 
to the evaluation, but his judgment might well be tempered by the opin- 
ions of others affected by the materials. The question of whether an 
instructional item will stimulate classroom discussion and search for 
further information is an item to be ''rated" in many forms. In this 
situation it would seem that information from students would be partic- 
ularly valuable. And when the question is raised with teachers, it 
would be well to give an opportunity to indicate reasons for a rating 
and perhaps to state the questions raised, if any, by the material. It 
may be that the material presents a case so badly that it will be a 
valuable vehicle for discussion. A rating from "superior" to "poor" 
cannot cope with such a contingency alone. 

As stated here and earlier, forms which supplement ratings or 
check lists with comments produce more complete evaluation information 
than do either rating forms or totally unstructured requests for opinion 
alone. Requests for comment must, however, be as "single" in intent as 
possible. One questionnaire received asks the following question: 

When critical controversial issues of the present 
are included in the material, are these issues presented in 
such a way as to lead toward better citizenship on the part 
of students? 



Any information elicited by that question consists of each responder's 
judgment as to what issues are critically controversial today, what 
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"better citizenship" is, and what type of presentation leads to that 
better citizenship. More useful information might be obtained by list- 
ing, perhaps at considerable length and in some detail, what a responsible 
authority (all department heads, a committee of the Board, or some such 
group) considers "the critical controversial issues of the present" and 
asking each responder to indicate whether each one is covered, whether he 
considers the coverage in each case thorough, fair, and properly aimed at 
its intended audience, and the reasons for his opinions. 

Some of the forms received were designed to help responders 
evaluate effectively by asking pertinent questions. Instead of merely 
asking for a check mark to indicate whether a teacher's manual is good, 
a form might ask questions about specific aspects of teacher's manuals: 

Are lessons planned with time sequences? Are suggested procedures easy 
to follow? Is there special emphasis on probable problem areas? Are 



supplementary drills, discussion questions, and outside activities sug- 



gested? 



Considerable difference in a responder's perspective might be 



engendered by asking for ratings and opinions in the context of compari- 
son as a means of getting an idea across: Is it important that the color 

version of the film be purchased: Will this tape hold the student's 

interest better than the teacher would? Is this new film so far superior 
to the one on hand as to warrant its purchase? Will a filmstrip teach 
this concept as well as a film will? Is this film too long to be prac- 



tical as a means of teaching this concept? 
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Implied in the questions above are others concerning costs. 

Costs of purchase, maintenance, and support all have their effect on a 
responder's judgment about a product. In the realm of books, films, 
and other software, the fact that there may be maintenance and support 
costs is not as obvious as in the realm of equipment. Nevertheless, 
flimsy books may have to be rebound, someone must rewind used tapes and 
films, elements must be sorted and returned to place in kits and displays, 
consumable workbooks must be replaced. Questionnaires can get at the 
facts about these points and also elicit some information as to their 
importance to a decision about a material. 

In equipment evaluations, central points should certainly be 
safety for students and teachers, availability of sufficient appropriate 
software, and the need for and availability of specially trained operating 
personnel. Very few forms used in evaluating equipment were received; 
they consisted primarily of tests of operating efficiency. Check lists 
could easily cover these central points as well as matters of maintenance 
and support costs, availability of service, durability, compatibility 
with support equipment already on hand, and so forth. 

Other questions which may affect purchase plans and which can 
easily be asked — though perhaps not always expressed in easily agreed- 
upon terms — deal with such matters as attractiveness and convenience. 

i 

Legibility of print material and ease of use of nonprint material can be 
investigated too. A considerable amount of valuable information can be 



obtained in a useful and relatively easily handled form by means of a 
check list or a questionnaire. 



APPENDIX C 



FORMS USED IN THE SURVEY 



Initial Letters 

Two very similar letters went to the representatives in the 
fifty states and the District of Columbia of the Association of Chief 
State School Audio-Visual Officers (ACSSAVO) and to the coordinators 
in the fifty states and the District of Columbia of projects under 
Title III of the National Defense Education Act. With the letters 
went descriptions of the sorts of projects about which information was 
sought, and a form post card for replying about telephone interview ar 
rangements. The letter to ACSSAVO members is reproduced on page 102. 

Telephone Interview Check List and Record 

On the basis of the post card replies, initial information 
was entered on a form (page 103) which facilitated scheduling and re- 
cording the interviews as well as serving as a means of ensuring that 
all relevant questions were asked and information given. 

Letter to Persons Recommended as interested in Selection 

A form letter (page 104) went to numbers of educators whose 
names were received in the telephone interviews or by correspondence. 
Adaptations of the letter were sent to certain individuals by the 
staff of the EPIE Institute, in cases where information on a specific 
project was sought, or when it was important to say specifically who 
had recommended that the letter be written. 
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Memorandum to Responders for Approval of Entries 

A form memorandum (page 106) went to persons who had supplied, 
specifically for this survey, information on projects written up as 
examples. A letter covering the same points, but also describing the 
survey to some extent, went to persons similarly responsible whose 
reports had come to the EPIE Institute independently of the survey. 
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THE UNIVERSITY OF THE STATE OF NEW YORK 
THE STATE EDUCATION DEPARTMENT 
ALBANY, NEW YORK 12224 



DIVISION OF EDUCATIONAL COMMUNICATIONS 
LEE E. CAMPION, DIRECTOR 
GR 4.9074 

June 1969 



Dear 



I The New York State Education Department, in cooperation with the education 

departments of California, Delaware, Illinois, New Jersey, North Carolina, 
Pennsylvania, and Texas, under an Office of Education Title V ESEA contract, 

| is compiling information on programs to evaluate instructional materials, 

equipment, and systems. The information will be consolidated into a "refer- 
| ence work," to be used as an aid in strengthening the role of State depart- 

ments in assisting in materials selection. We shall see that you receive a 
| copy of the reference work, which I hope you will find helpful, 

l 

| The task of preparing the reference work we have subcontracted to Educational 

j Products Information Exchange (EPIE) Institute, specialists in instructional 

I materials descriptions and evaluations. They would like to discuss the pro- 

| ject with you, as someone knowledgeable about product evaluation activities 

in your state, so they propose to telephone you during the week of 
I if that is convenient for you. They will ask you for copies of or references 

s to reports -- formal or informal -- on evaluation programs you have been con- 

i cerned with, and for names of persons who may be able to supply information 

I on other materials evaluation projects. The enclosed article on "Project Aim," 

l and the short excerpt from a report on a study of video tape recorders, show 

f the types of programs we are interested in. These were done by State depart- 

ments, but we know that similar approaches to materials evaluation have been 
j undertaken by Title III centers, or under Title I or Title II auspices, or by 

! local school systems, and we hope you can lead us to such programs in your state, 

t 



I Again, a project researcher plans to call you during the week of 

If you will be away, or if that does not give you time to get ready for the 
interview, please inform EPIE of your wishes on the enclosed card. Or, if you 
think we should discuss the matter with someone else rather than you, will you 
please refer this letter and its enclosures to him and drop EPIE a note giving 
his name? Their address is 386 Park Avenue South, New York, New York 10016, 

j Thank you very much for whatever help you can give us. 



Sincerely yours, 




LEC/td 
Encs • 



Lee E, Campion 
Director 
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DATE: 






NO: 



We are interested in projects in which instructional materials, equipment, and systems were 
(or are being )evaluated as an aid in selection. We are starting at State level but are inter- 
ested in any such project. NOT just audiovisual — ANY material, equipment, system. 

1. What ones ha-^you been connected with? 



1 

4 



\ 
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l 



2. Can you send reports — articles, memos, whatever form? 
and return original . Or can you give us references? 



If you want, we will Xerox 



| 






- 



3. Who else should we call — 
State? 

local? 



university? 

Title I E8EA? 

Title II ESEA? 

Title III ESEA? 

testing organization (eg, Buyer* s Lab)? 



other? 



4. You will be sent a copy of the reference work that comes out of this survey 
It may generate requests for information from you. 
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THE UNIVERSITY OF THE STATE OF NEW YORK 
THE STATE EDUCATION DEPARTMENT 
ALBANY, NEW YORK 12224 



DIVISION OF EDUCATIONAL COMMUNICATIONS 
LEE E. CAMPION, DIRECTOR 
GR 4*5974 



Dear 



The New York State Education Department, in cooperation with the 
education departments of California, Delaware, Illinois, New Jersey, 

North Carolina, Pennsylvania, and Texas, under an Office of Education 
Title V ESEA contract, is compiling information on programs to evaluate 
instructional materials, equipment, and systems. The information will 
be consolidated into a "reference work, " to be used as an aid in 
strengthening the role of State departments in assisting in materials 
selection. We shall see that you receive a copy of the reference work, 
which I hope you will find helpful. 

The task of preparing the reference work we have subcontraced to 
Educational Products Information Exchange (EPIE) Institute, specialists 
in instructional materials descriptions and evaluations. We have heard 
from an educator in your state that you may be able to supply EPIE with 
infromation about one or more programs designed to evaluate instructional 
materials or equipment — textbooks, films, tapes, kits, slides, pro- 
jectors, or other such material — for purposes of assisting in the 
selection of such materials for classroom use. If that is the case, and 
some sort of documentation of any such project exists, it would be most 
helpful if you could supply copies of any such documentation — a report 
on the project, perhaps, or procedures and forms used in it, or any such 
pertinent information. If copies are scarce, your are welcome to 
instruct EPIE to make copies of your material and return the originals 
to you. 

If you can refer EPIE to others in your state who may have informa- 
tion about programs such as I have described, it would be very much 
appreciated • 
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May I request that you address materials or any correspondence to 
the EPIE Institute, 386 Park Avenue South, New York, New York 10016. 

It will expedite matters if you mark the mail for "Title V Materials 
Evaluation Survey. " 



Thank you very 



I 

[■ 



much for whatever help you can give us. 

Sincerely yours. 






Lee E. Campion. 
Di rector 
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M the educational products information exchange institute 

386 PARK AVENUE SOUTH, NEW YORK, NEW YORK 10016 • A Nonprofit Professionals’ Cooperative 



Memorandum for 
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Subject: Report, "Evaluation Practices Used in the Selection of Educational 

Materials and Equipment" 

From: Betty Braxton Preston Date: October 20, 1969 



The staff of the EPIE Institute has completed the first draft of the 
report on selection of educational materials which we undertook under a sub- 
contract with eight states (California, Delaware, Illinois, New Jersey, New 
York, North Carolina, Pennsylvania, and Texas), pursuant to their contract, 
under Title V ESEA, with the U. S. Office of Education. The draft takes the 
form primarily of descriptions of selection practices supplied to us during 

the course of our inquiry. 



You were kind enough to supply a description of a selection practice and 
I enclose herewith the portion of the draft which is based on your description. 
Will you please review the draft and let me know, just as soon as you conveni- 
ently can, whether we have reproduced the information accurately? We are most 
anxious to be sure that our report reflects exac.ly the various selection prac- 
tices we describe. 



We shall look forward to a reply from you very soon. If we have not heard 
by November 14, we shall assume that the draft is satisfactory to you. 



One more point: When the report i 

you. We very much hope that it will 
haps be of service in your continued 



s completed, a copy of it will be sent to 
interest you and your colleagues and per- 
selection efforts. 



Thank you very much! 



